The revolutionary induced pluripotent stem cell (iPSC) technology provides a new means for cell replacement therapies and drug screening. Small molecule compounds have been found extremely useful to improve the generation of iPSCs and understand the reprogramming mechanism. Here we report the identification of a novel chemical, CYT296, which improves OSKM-mediated induction of iPSCs for >10 folds and enables efficient reprogramming with only Oct4 in combination with other small molecules. The derived iPSCs are genuinely pluripotent and support the development of two ‘All-iPSC’ mice by tetraploid complementation. CYT296 profoundly impacts heterochromatin formation without affecting cell viability. MEFs treated with CYT296 exhibit de-condensed chromatin structure with markedly reduced loci containing heterochromatin protein 1α (HP1α) and H3K9me3, which is very similar to the chromatin configuration in embryonic stem cells (ESCs). Given that an open chromatin structure serves as a hallmark of pluripotency and has to be acquired to fulfill reprogramming, we propose that CYT296 might facilitate this process by disrupting condensed chromatin, thereby creating a more favorable environment for reprogramming. In agreement of this idea, shRNA targeting HP1α also promotes the generation of iPSCs. Thus current findings not only provide a novel chemical for efficient iPSC induction, but also suggest a new approach to regulate somatic cell reprogramming by targeting chromatin de-condensation with small molecules.